Messaging Routine Interruption and Synchronization

ABSTRACT

A requesting program makes a first call to a messaging routine. The first call provides at least one request for at least one requested messaging operation to be performed by the messaging routine. The requesting program is permitted to continue execution prior to completion of the at least one requested messaging operation rather than blocking in response to the first call. The messaging routine is interrupted prior to completion of the at least one requested messaging operation in response to a second call which is made from the requesting program to a synchronization routine. Calling the synchronization routine has an effect of interrupting any outstanding call to the messaging routine. In some cases, an application program interface containing the synchronization routine has functionality supporting serialization of calls to the messaging routine. The call to the messaging routine is serialized, and execution of the program continues.

COPYRIGHT AUTHORIZATION

A portion of the disclosure of this patent document contains materialwhich is subject to copyright protection. The copyright owner has noobjection to the facsimile reproduction by anyone of the patent documentor the patent disclosure, as it appears in the Patent and TrademarkOffice patent file or records, but otherwise reserves all copyrightrights whatsoever.

RELATED APPLICATION

The present application claims priority to, and incorporates byreference the entirety of, U.S. patent application Ser. No. 13/082,193filed Apr. 7, 2011.

BACKGROUND

Message Passing Interface (MPI) is a group of language-independentApplication Program Interface (API) specifications which defineprotocols for software processes to communicate with one another bysending and receiving messages. MPI is a de facto standard, unsanctionedat present by any major standards body. A variety of MPI specificationversions and MPI implementations exist, many of which are directed atmulti-threaded and/or parallel programs. MPI is widely used inHigh-Performance Computing (HPC) environments.

SUMMARY

In some embodiments, a requesting program makes a first call to amessaging routine. The first call provides at least one request for atleast one requested messaging operation to be performed by the messagingroutine. The requesting program is permitted to continue execution priorto completion of the at least one requested messaging operation ratherthan blocking in response to the first call. The messaging routine isinterrupted prior to completion of the at least one requested messagingoperation in response to a second call which is made from the requestingprogram to a synchronization routine. Calling the synchronizationroutine has an effect of interrupting any outstanding call to themessaging routine. In some embodiments, an application program interfacecontaining the synchronization routine has functionality supportingserialization of calls to the messaging routine.

In some embodiments, an application program interface contains asynchronization routine. A program makes a call to a messaging routineproviding at least one request for at least one requested messagingoperation to be performed by the messaging routine. The programcontinues execution prior to completion of the at least one requestedmessaging operation rather than being blocked by the call to themessaging routine. The program subsequently makes a call to thesynchronization routine, which results in an interruption of themessaging routine prior to completion of the at least one requestedmessaging operation. The call to the messaging routine is serialized,and execution of the program continues.

The examples given are merely illustrative. This Summary is not intendedto identify key features or essential features of the claimed subjectmatter, nor is it intended to be used to limit the scope of the claimedsubject matter. Rather, this Summary is provided to introduce—in asimplified form—some concepts that are further described below in theDetailed Description. The innovation is defined with claims, and to theextent this Summary conflicts with the claims, the claims shouldprevail.

DESCRIPTION OF THE DRAWINGS

A more particular description will be given with reference to theattached drawings. These drawings only illustrate selected aspects andthus do not fully determine coverage or scope.

FIG. 1 is a block diagram illustrating a computer system having multipleprocessors, memory, software threads, an MPI library, and other items inan operating environment which may be present on multiple network nodes,and also illustrating configured storage medium embodiments;

FIG. 2 is a block diagram illustrating asynchronous callback andinterruptible wait aspects of an enhanced MPI library in an examplearchitecture;

FIG. 3 is a flow chart illustrating steps of some process and configuredstorage medium embodiments;

FIG. 4 is a sequence diagram illustrating interaction flow for anasynchronous callback request in some embodiments;

FIG. 5 is a sequence diagram illustrating interaction flow for anasynchronous callback response in some embodiments;

FIG. 6 is a sequence diagram illustrating interaction between ranks ofan application in some embodiments;

FIG. 7 is a sequence diagram illustrating interaction flow for aninterruptible wait request in some embodiments; and

FIG. 8 is a sequence diagram illustrating interaction flow for aninterruptible wait response in some embodiments.

DETAILED DESCRIPTION

Overview

Although MPI is widely used, available MPI approaches could be enhancedin helpful ways. Two kinds of MPI enhancements presented here arereferred to generally as “asynchronous callback” enhancements and as“interruptible wait” enhancements, respectively. Each may be added andused without the other, or they may both be added and used in a givenembodiment.

With regard to asynchronous callback, familiar MPI approaches lacksuitable support for callback driven completion notifications for MPIrequests. Users instead explicitly test or wait on a non-blockingrequest to detect completion. Callback driven completion notificationsare problematic because they would traditionally occur in the context ofan existing MPI call, restricting what calls the application can performin the callback. An MPI implementation could use threads to invoke thecallback, but this might break applications by making a single threadedapplication suddenly multi-threaded. Accordingly, asynchronous callbackenhancements like those provided herein offer a helpful alternative.

With regard to interruptible wait, familiar MPI specifications providetwo modes of checking for non-blocking request completion: blocking onone or more requests via the MPI_Wait series of routines (MPI_Wait,MPI_Waitsome, MPI_Waitany, MPI_Waitall), or testing for completion ofone or more requests via the MPI_Test series of routines (MPI_Test,MPI_Testsome, MPI_Testany, MPI_Testall.) The MPI_Wait calls all blockthe calling thread until the wait condition is satisfied. The MPI_Testcalls check for request completion but do not block the calling thread.In a multi-threaded MPI application or other implementation using theMPI_THREAD_SERIALIZED threading model, threads normally should not blockindefinitely, and thus they call MPI_Iprobe or one of the MPI_Testfunctions in a loop to check for incoming messages while still remainingresponsive to other threads in the application. As a result, one mightinsist that MPI implementations support MPI_THREAD_MULTIPLE, allowingconcurrent calls by multiple threads. However, the locking overhead ofimplementing thread safety may regress performance for single threadedapplications or applications that already implement their own threadserialization techniques. Accordingly, interruptible wait enhancementslike those provided herein offer a helpful alternative.

Some embodiments described herein may be viewed in a broader context.For instance, concepts such as asynchrony, callbacks, completion,messaging, interruption, or serialization may be relevant to aparticular embodiment. However, it does not follow from the availabilityof a broad context that exclusive rights are being sought herein forabstract ideas; they are not. Rather, the present disclosure is focusedon providing appropriately specific embodiments. Other media, systems,and methods involving asynchrony, callbacks, completion, messaging,interruption, and/or serialization are outside the present scope.Accordingly, vagueness and accompanying proof problems are also avoidedunder a proper understanding of the present disclosure.

With respect to asynchronous callback, some embodiments described hereinintroduce non-blocking send and receive functions, MSMPI_Isend_apc andMSMPI_Irecv_apc, that allow one to avoid waiting for completionexplicitly. Instead, any thread making progress that completes anoutstanding asynchronous procedure request, such as an APC request,queues a user-mode asynchronous callback (such as an APC) to theinitiating thread. The asynchronous callback executes outside of thecontext of any MPI call whenever the requesting thread enters analertable state. As used herein, “APC” refers to a particular kind ofasynchronous routine, namely, one provided in some Microsoft® operatingenvironments, which executes asynchronously in the context of aparticular thread. The use of an alertable state is linked in someenvironments to APCs, but is conceptually separable from the use ofasynchronous routines which execute in the context of a particularthread.

Unlike traditional callback models, these asynchronous callbackenhancements provide one or both of the following aids. First, thethread that initiated the request executes the callback, regardless ofwhich thread detected the request as being complete. This gives theapplication a deterministic thread context in which the callback runs,something traditional callback models fail to provide. Second, thethread that initiates the request controls when the callbacks canexecute by putting itself in a receptive (e.g., Alertable wait) state,allowing the application to delay executing callbacks until it isoutside of logically related sections of code.

With respect to interruptible wait, some embodiments described hereinintroduce a wait function, MSMPI_Waitsome_interruptible, that allows anapplication to make strong progress without the potential for deadlock.To complement this function, and to allow the client application tointerrupt an outstanding call to MSMPI_Waitsome_interruptible, someembodiments also introduce the synchronization functionsMSMPI_Queuelock_acquire and MSMPI_Queuelock_release. Thesesynchronization functions allow applications to implement properserialization to MPI calls between threads while at the same timeproperly interrupting any outstanding MSMPI_Waitsome_interruptiblecalls.

Allowing threads to interrupt another thread's blocking call allowshybrid/multithreaded applications to make strong progress while stillbeing responsive to local messaging requirements. Providingserialization and wait interruption as an atomic operation helpsapplications avoid potential race conditions between threads trying toaccess the MPI implementation. This design allows applications to usemultithreading while at the same time giving higher performance thancould likely be achieved with MPI_THREAD_MULTIPLE support. Theenhancement may also eliminate the overhead of internal MPI threads andpotential oversubscription of cores of an MPI implementation that usesone or more internal progress threads.

Reference will now be made to exemplary embodiments such as thoseillustrated in the drawings, and specific language will be used hereinto describe the same. But alterations and further modifications of thefeatures illustrated herein, and additional applications of theprinciples illustrated herein, which would occur to one skilled in therelevant art(s) and having possession of this disclosure, should beconsidered within the scope of the claims.

The meaning of terms is clarified in this disclosure, so the claimsshould be read with careful attention to these clarifications. Specificexamples are given, but those of skill in the relevant art(s) willunderstand that other examples may also fall within the meaning of theterms used, and within the scope of one or more claims. Terms do notnecessarily have the same meaning here that they have in general usage,in the usage of a particular industry, or in a particular dictionary orset of dictionaries. Reference numerals may be used with variousphrasings, to help show the breadth of a term. Omission of a referencenumeral from a given piece of text does not necessarily mean that thecontent of a Figure is not being discussed by the text. The inventorasserts and exercises his right to his own lexicography. Terms may bedefined, either explicitly or implicitly, here in the DetailedDescription and/or elsewhere in the application file.

As used herein, a “computer system” may include, for example, one ormore servers, motherboards, processing nodes, personal computers(portable or not), personal digital assistants, cell or mobile phones,other mobile devices having at least a processor and a memory, and/orother device(s) providing one or more processors controlled at least inpart by instructions. The instructions may be in the form of firmware orother software in memory and/or specialized circuitry. In particular,although it may occur that many embodiments run on workstation or laptopcomputers, other embodiments may run on other computing devices, and anyone or more such devices may be part of a given embodiment.

A “multithreaded” computer system is a computer system which supportsmultiple threads of execution. The term “thread” should be understood toinclude any code capable of or subject to scheduling (and possibly tosynchronization). In some cases, a thread may also be known by anothername, such as “task,” “process,” or “coroutine,” for example. In somecases, such as many MPI development contexts, a distinction is madebetween a “thread” and a “process” in that a process may have multiplethreads. In general, threads may run in parallel, in sequence, or in acombination of parallel execution (e.g., multiprocessing) and sequentialexecution (e.g., time-sliced). Multithreaded environments have beendesigned in various configurations. Execution threads may run inparallel, or threads may be organized for parallel execution butactually take turns executing in sequence. Multithreading may beimplemented, for example, by running different threads on differentcores in a multiprocessing environment, by time-slicing differentthreads on a single processor core, or by some combination oftime-sliced and multi-processor threading. Thread context switches maybe initiated, for example, by a kernel's thread scheduler, by user-spacesignals, or by a combination of user-space and kernel operations.Threads may take turns operating on shared data, or each thread mayoperate on its own data, for example.

A “logical processor” or “processor” is a single independent hardwarethread-processing unit, such as a core in a simultaneous multithreadingimplementation. As another example, a hyperthreaded quad core chiprunning two threads per core has eight logical processors. Processorsmay be general purpose, or they may be tailored for specific uses suchas graphics processing, signal processing, floating-point arithmeticprocessing, encryption, I/O processing, and so on.

A “multiprocessor” computer system is a computer system which hasmultiple logical processors. Multiprocessor environments occur invarious configurations. In a given configuration, all of the processorsmay be functionally equal, whereas in another configuration someprocessors may differ from other processors by virtue of havingdifferent hardware capabilities, different software assignments, orboth. Depending on the configuration, processors may be tightly coupledto each other on a single bus, or they may be loosely coupled. In someconfigurations the processors share a central memory, in some they eachhave their own local memory, and in some configurations both shared andlocal memories are present.

“Kernels” include operating systems, hypervisors, virtual machines, BIOScode, and similar hardware interface software.

“Code” means processor instructions, data (which includes constants,variables, and data structures), or both instructions and data.

A “routine” can be a function (which returns a value) or a procedure(which does not). Routines are an example of code.

“Calling” and “invoking” are used interchangeably herein with regard toroutines.

An “API” is an interface to one or more routines, which specifies atleast the parameter(s) to be passed when calling the routine(s).

A “library” is a collection of routines which can be invoked from othercode. A library can export routines through an API, for example, to becalled by one or more threads.

An “MPI standard” is any version of any standard or MPI specificationpublished by the Message Passing Interface Forum.

A “message passing interface library” is a library which implements atleast one message passing routine, such as for example a routine definedin at least one MPI standard, or another routine for communicationbetween threads by way of message passing.

“Program” is used broadly herein, to include applications, kernels,drivers, interrupt handlers, libraries, and other code written byprogrammers (who are also referred to as developers).

“Automatically” means by use of automation (e.g., general purposecomputing hardware configured by software for specific operationsdiscussed herein), as opposed to without automation. In particular,steps performed “automatically” are not performed by hand on paper or ina person's mind; they are performed with a machine. However,“automatically” does not necessarily mean “immediately”.

A “lock” is a flag, counter, queue, semaphore, mutex, or other programobject that negotiates mutual exclusion among threads. Locks are used tocontrol thread access to a common resource, e.g., to serialize access toa resource that is used by multiple threads.

Throughout this document, use of the optional plural “(s)” means thatone or more of the indicated feature is present. For example,“thread(s)” means “one or more threads” or equivalently “at least onethread”.

Throughout this document, unless expressly stated otherwise anyreference to a step in a process presumes that the step may be performeddirectly by a party of interest and/or performed indirectly by the partythrough intervening mechanisms and/or intervening entities, and stilllie within the scope of the step. That is, direct performance of thestep by the party of interest is not required unless direct performanceis an expressly stated requirement. For example, a step involving actionby a party of interest such as “accessing”, “associating”, “calling”,“detecting”, “displaying”, “interrupting”, “invoking”, “sending”,“transmitting”, or “utilizing” with regard to a destination or othersubject may involve intervening action such as forwarding, copying,uploading, downloading, encoding, decoding, compressing, decompressing,encrypting, decrypting, authenticating, communicating, and so on by someother party, yet still be understood as being performed directly by theparty of interest.

Whenever reference is made to data or instructions, it is understoodthat these items configure a computer-readable memory therebytransforming it to a particular article, as opposed to simply existingon paper, in a person's mind, or as a transitory signal on a wire, forexample. A computer-readable medium is presumed to not be merely apropagated signal unless expressly stated otherwise.

Operating Environments

With reference to FIG. 1, an operating environment 100 for an embodimentmay include a computer system 102. The computer system 102 may be amultiprocessor computer system, or not. An operating environment mayinclude one or more machines in a given computer system, which may beclustered, client-server networked, and/or peer-to-peer networked. Anindividual machine is a computer system, and a group of cooperatingmachines is also a computer system. A given computer system 102 may beconfigured for end-users, e.g., with applications, for administrators,as a server, as a distributed processing node, and/or in other ways.

Human users 104 may interact with the computer system 102 by usingdisplays, keyboards, and other peripherals 106. System administrators,developers, engineers, and end-users are each a particular type of user104. Automated agents acting on behalf of one or more people may also beusers 104. Storage devices and/or networking devices may be consideredperipheral equipment in some embodiments. Other computer systems notshown in FIG. 1 may interact with the computer system 102 or withanother system embodiment using one or more connections to a network 108via network interface equipment, for example.

The computer system 102 includes at least one logical processor 110, andgenerally includes multiple processors 110 as shown in the example ofFIG. 1. The computer system 102, like other suitable systems, alsoincludes one or more computer-readable storage media 112. Media 112 maybe of different physical types. The media 112 may be volatile memory,non-volatile memory, fixed in place media, removable media, magneticmedia, optical media, and/or other media (as opposed to merely asignal). In particular, a configured medium 114 such as a CD, DVD,memory stick, or other removable non-volatile memory medium may becomefunctionally part of the computer system when inserted or otherwiseinstalled, making its content accessible for use by processor 110. Theremovable configured medium 114 is an example of a computer-readablestorage medium 112. Some other examples of computer-readable storagemedia 112 include built-in RAM, ROM, hard disks, and other storagedevices which are not readily removable by users 104.

The medium 114 is configured with instructions 116 that are executableby a processor 110; “executable” is used in a broad sense herein toinclude machine code, interpretable code, and code that runs on avirtual machine, for example. The medium 114 is also configured withdata 118 which is created, modified, referenced, and/or otherwise usedby execution of the instructions 116. The instructions 116 and the data118 configure the medium 114 in which they reside; when that memory is afunctional part of a given computer system, the instructions 116 anddata 118 also configure that computer system. In some embodiments, aportion of the data 118 is representative of real-world items such asproduct characteristics, inventories, physical measurements, settings,images, readings, targets, volumes, and so forth. Such data is alsotransformed by asynchronous callback and/or interruptible wait asdiscussed herein.

An application 120 having threads 122 organized in process ranks 124, amessage passing interface library 126 with routines 128 havingsignatures 130 and with a progress engine 132, and other items shown inthe Figures and/or discussed in the text may reside partially orentirely within one or more media 112, thereby configuring those media.An operating environment may also include a display 134 and otherhardware, such as buses, power supplies, and accelerators, for instance.

High Performance Computing (HPC) support 136 in the form of a kernel,parallel programming development tools, and/or parallel processinghardware, for example, may be present. A given operating environment 100may include an Integrated Development Environment (IDE) 138 whichprovides a developer with a set of coordinated software developmenttools. In particular, some of the suitable operating environments forsome embodiments include or help create a Microsoft® Visual Studio®development environment (marks of Microsoft Corporation) configured tosupport HPC program development.

One or more items are shown in outline form in FIG. 1 to emphasize thatthey are not necessarily part of the illustrated operating environment,but may interoperate with items in the operating environment asdiscussed herein. It does not follow that items not in outline form arenecessarily required, in any Figure or any embodiment.

Systems

FIG. 2 illustrates an architecture which is suitable for use with someembodiments. An enhanced MPI library 126 includes a completion-awaitingroutine 202 which has a completion condition 204 including a list 206 ofmessaging operations 208. Upon the completion-awaiting routine's return,a completion indication 210 indicates the extent and in some cases thedetails of the satisfaction (or lack thereof) of the completioncondition 204.

More generally, some embodiments include messaging requests 212 createdand processed within an MPI architecture 214 which includes threads 122and an MPI library 126. In some MPI architectures 214, threads ofinterest belong to an application 120 and are assigned a role as acommunication thread 216 or as a (non-MPI) computational worker thread218. Threads 122 are identified by thread identifiers 220. In someembodiments, threads can be serialized using a lock 222, which mayinclude one or more mutexes, flags, or other individual variables.

In some embodiments, an asynchronous callback 224 is associated with athread 122, e.g., by using a broad callback association routine 226 thatassociates a given callback with any of a predetermined group of severallibrary 126 routines 128, or by using a narrow callback associationroutine 228 that associates a given callback with a predeterminedspecific library 126 routine 128 (in which case different routines 128would have different respective narrow callback association routines228). The callback 224 will execute in its associated thread's context230. In some cases, callbacks can execute only when a thread's state 232is open to callback execution, as specified in a thread state indicator234.

With regard to asynchronous callback, a familiar pattern used in somehybrid MPI/multi-threaded applications 120 is that each rank 124 has oneMPI communication thread 216 and multiple non-MPI worker threads 218.The number of worker threads is often N−1, where N is the number ofcores on a host or NUMA (non-uniform memory access/non-uniform memoryarchitecture) node. The threads interact through message queues. Forinstance, one queue may hold requests from worker threads like “fetch meblock 51” when data block 51 is owned by another rank, while anotherqueue holds replies like “send requested block 51”. Each rank 124 mayrequest some blocks and satisfy block requests from others. Thecommunication thread 216 deals asynchronously with both the MPI library126 and other threads 122. A familiar message flow may thus span acrossnodes (A, B, . . . ) as well as threads (X, Y, . . . where X is thecommunication thread): e.g., A:Z→A:X→B:X→B:Y→B:X→A:X→A:Z. With someembodiments using asynchronous callback enhancements, such a messageflow is modified to become A:Z→B:X→B:Y→A:Z.

In some embodiments, an MPI implementation processes outstandingrequests 212 in the context of application calls into the MPI library126. The application 120 thus controls the thread context 230 in whichI/O progress is made. An MPI standard states that progress is made onoutstanding requests when the application requests completion of anyrequest (e.g., MPI Standard version 2.2, section 3.7, page 57, citedhere for background information only and not for incorporation byreference.) When an application initiates a non-blocking send (e.g.,using an MSMPI_Isend_apc routine 128) or receive (e.g., using anMSMPI_Irecv_apc routine 128), one MPI implementation (denoted hereMSMPI) stores the requesting thread information 220 to allow it to queuea Windows Asynchronous Procedure Call (APC) callback 224 to therequesting thread when that request completes, independently from anyexplicit call to complete such requests. APCs are exposed toapplications through the QueueUserAPC function. Different asynchronouscallbacks 224 may be used in other implementations.

With regard to interruptible wait, an MPI standard supports multiplelevels of threading in MPI implementations, as described in thefollowing table:

MPI_THREAD_SINGLE The application may only have a single thread.MPI_THREAD_FUNNELED The application must make all MPI calls from asingle thread MPI_THREAD_SERIALIZED The application must serialize callsto MPI, and only a single thread may call MPI at a time.MPI_THREAD_MULTIPLE The application may make concurrent calls to MPIfrom any thread.

Like some other implementations, the MSMPI implementation supports up toMPI_THREAD_SERIALIZED, so multithreaded MPI applications serialize theiraccess to MPI when using MSMPI. It will be understood by one of skillthat implementing MPI_THREAD_MULTIPLE support is a large undertaking,with significant potential for regressing single-threaded performance.

In a familiar MPI architecture with hybrid MPI/multi-threadedapplications as discussed above, the communication thread 216 does notuse any blocking MPI calls because doing so would risk making thecommunication thread 216 unresponsive to requests from the computethreads 218. A familiar workaround is for the communication thread toloop, alternating between (a) calling MPI_Iprobe/MPI_Test to detectincoming messages and (b) checking the thread message queues. When amessage is received from MPI, it is dispatched to the appropriatecompute thread for processing. When a compute thread needs to requestdata or send the response to a request, it queues the outbound messageto the communication thread. The communication thread then initiates alltransfers on behalf of the compute threads. A result of this traditionaldesign pattern is that a thread essentially busy-waits checking for I/Owork to be done. If there is no I/O work to do, the thread ends upwasting a whole CPU. But with interruptible wait enhancements, thecommunication thread 216 is able to make strong progress (blocked in theMPI progress engine) while still being responsive to other threads thatwould make their own MPI calls or queue outbound messages to thecommunication thread.

With reference to FIGS. 1 and 2, some embodiments provide a computersystem 102 with a logical processor 110 and a memory medium 112configured by circuitry, firmware, and/or software to transform an MPIlibrary and/or an MPI architecture by extending functionality withasynchronous callback enhancements and/or interruptible waitenhancements as described herein.

Some asynchronous callback embodiments include a computer system 102with at least two logical processors 110, each logical processor inoperable communication with a memory 112. At least two threads 122reside in (and thus configure) the memory 112. An MPI messaging request212 residing in the memory has a requesting thread identifier 220 whichidentifies one of the threads. The messaging request 212 also has acompletion indicator 210 which indicates whether the messaging requesthas completed. An asynchronous callback 224 residing in the memory has adeterministic thread callback context 230, which associates the callbackwith the thread identified in the messaging request, thereby determiningthe thread in which the callback will execute.

In some embodiments, the thread identified in the messaging request alsohas a state indicator 234 which indicates one of the following when thethread is executing: a closed state 232 in which the thread will notexecute asynchronous callbacks, an open state 232 in which the threadwill execute asynchronous callbacks.

In some embodiments, the system includes an application 120 having atleast two ranks 124 of processes. Each rank has a communication thread216 and multiple worker threads 218. The thread identified in themessaging request 212 is a communication thread of one of the ranks.

In some embodiments, the memory contains a broad callback associationroutine 226, namely, a routine which is configured to set thedeterministic thread callback context 230 (and often the callback aswell) for any kind of non-blocking messaging request 212 that isrecognized in the system. A routine is thus configured by virtue ofstructure, namely, data 118 and/or instructions 116 whose behavior canbe accurately predicted by one of skill given its source code and adescription of the system.

In some embodiments, the memory contains a narrow callback associationroutine 228. For example, one routine 228 sets the deterministic threadcallback context for a persistent messaging request, one routine 228sets context 230 for a synchronous send messaging request, anotherroutine 228 sets context 230 for a ready send messaging request, and yetanother routine 228 sets context 230 for a buffered send messagingrequest.

Building on some of the foregoing embodiments, in some cases the memorycontains a message passing interface library 126 that is configured toprocess the messaging request 212 and to set the completion indicator210 to indicate that the messaging request has completed. Thedeterministic thread callback context 230 in some embodiments includes aqueue in which the callback is queued for execution by the threadidentified in the messaging request.

Some interruptible wait embodiments include a computer system 102 withat least two logical processors 110, each logical processor in operablecommunication with a memory 112. At least two threads 122 reside in thememory 112. A message passing interface library progress engine 132 alsoresides in the memory, as does an unsatisfied completion condition 204specifying a list 206 of messaging operation(s) 208 which have not yetbeen completed. An interruptible completion-awaiting routine 202 of amessage passing interface library is also present in memory. Theinterruptible completion-awaiting routine 202 is configured (by itsstructure in the form of data and/or instructions) to returnsuccessfully from a blocked condition in the absence of interruptionwhen the completion condition is satisfied.

In some embodiments, a completion-awaiting routine can be viewed as anenhancement of familiar MPI wait and waitsome routines. Some examples ofunenhanced completion-awaiting routines are the familiar MPI_Wait( ) andMPI_Waitsome( ) routines. A completion-awaiting routine is configured toreturn successfully from a blocked condition after a specified messagepassing completion condition is satisfied. The completion condition mayvary. For example, one MPI_Waitsome( ) routine may be configured toreturn success after at least one of a specified list of MPI send orreceive operations completes, while another one MPI_Waitsome( ) routineis configured to return success after at least N (N>1) operationscomplete, and a third MPI_Waitsome( ) routine is configured to returnsuccess only after all of the operations complete. An MPI standarddefines MPI_Waitsome as completing when at least one request completes.It can return with more than one completed request, unlike MPI_Waitanywhere only one request is complete at the time the call returns.

In a given embodiment, one or more enhanced MPI_Wait routines could beinterrupted. Some embodiments focus on MPI_Waitsome because its familiarversion has an output parameter indicating the number of requests thatactually completed. In some embodiments, theMSMPI_Waitsome_interruptible routine 202 has the same function signature130 as MPI_Waitsome, making it easier for some developers to grasp. Theother familiar wait routines arguably don't support returning zerocompletions as naturally: MPI_Waitany always returns with 1, indicatingwhich of the input requests completed. MPI_Waitall returns when all havecompleted, with no output indicating how many (it's implicitly ‘N’).MPI_Wait only waits for a single request, and has no output parameters.In some embodiments, MPI error values may be leveraged for indicating aninterruption, although by default some MPI implementations abort theprocess on any error unless overridden explicitly by the application.

However, any permutation of “wait-family” call(s) could be enhancedconsistent with the teachings herein, even if the enhanced API signature130 didn't match the non-interruptible familiar version. One way todescribe the set of possible wait-family routines (waitsome, waitany,etc.) is according to what request(s) have completed upon a normalreturn:

(a) At least 1 of a list of N has completed

(b) Exactly 1 of a list of N has completed

(c) At least 2 of a list of N have completed

(d) Exactly 2 of a list of N have completed

(e) Etc., up to at least N of a list of N/exactly N of a list of N havecompleted

The foregoing contemplates all logical possibilities, even if only someof them happen to be part of any current MPI standard. All of theseroutines may be enhanced as completion-awaiting routines whoseinterruptible version returns with zero completions. It may happen thatsome enhanced completion-awaiting routines will be more frequently used,such as an interruptible version of MPI-waitsome, perhaps.

In some embodiments, one of the threads is a communication thread 216 ofan application 120 having at least two ranks 124 of processes, and theother thread is a worker thread 218 of the application. Each processrank 124 has a communication thread and multiple worker threads in thisembodiment.

Some embodiments include a lock 222 for use in serializing thread callsto the message passing interface library. Routines implementing a FIFOqueue lock are discussed below as an example, but other embodiments mayuse different locks 222.

In some embodiments, the system 102 has an MPI architecture 214 thatcomplies with a standard description such as MPI_THREAD_FUNNELED and/orMPI_THREAD_SERIALIZED. In some, the completion-awaiting routine 202 isenhanced with interruptible wait functionality but has the same functionsignature 130 as a standard MPI routine.

Some embodiments include both asynchronous callback and interruptiblewait enhancements. For example, one system builds on one or more of theforegoing interruptible wait embodiments and also includes a messagingrequest 212 residing in the memory and having a requesting threadidentifier 220 which identifies one of the threads, and an asynchronouscallback 224 residing in the memory. The callback 224 has adeterministic thread callback context 230 which associates the callbackwith the thread identified in the messaging request, thereby determiningthe thread in which the callback will execute. In some embodiments, thethread identified in the messaging request also has a state indicator234 which indicates one of the following when the thread is executing: aclosed state 232 in which the thread will not execute asynchronouscallbacks, an open state 232 in which the thread will executeasynchronous callbacks.

In some embodiments peripherals 106 such as human user I/O devices(screen, keyboard, mouse, tablet, microphone, speaker, motion sensor,etc.) will be present in operable communication with one or moreprocessors 110 and memory. However, an embodiment may also be deeplyembedded in a system, such that no human user 104 interacts directlywith the embodiment. Software processes may be users 104.

In some embodiments, the system includes multiple computers connected bya network. Networking interface equipment can provide access to networks108, using components such as a packet-switched network interface card,a wireless transceiver, or a telephone network interface, for example,will be present in a computer system. However, an embodiment may alsocommunicate through direct memory access, removable nonvolatile media,or other information storage-retrieval and/or transmission approaches,or an embodiment in a computer system may operate without communicatingwith other computer systems.

Some embodiments operate in a “cloud” computing environment and/or a“cloud” storage environment in which computing services are not ownedbut are provided on demand. For example, processes of ranks 124 and MPIlibrary routines 128 may run on multiple devices/systems 102 in anetworked cloud.

Processes

FIG. 3 illustrates some process embodiments in a flowchart 300.Processes shown in the Figures may be performed in some embodimentsautomatically, e.g., by an application 120 under control of a script orotherwise requiring little or no contemporaneous user input as itexecutes in an enhanced MPI architecture 214. Processes may also beperformed in part automatically and in part manually unless otherwiseindicated. In a given embodiment zero or more illustrated steps of aprocess may be repeated, perhaps with different parameters or data tooperate on. Steps in an embodiment may also be done in a different orderthan the top-to-bottom order that is laid out in FIG. 3. Steps may beperformed serially, in a partially overlapping manner, or fully inparallel. The order in which flowchart 300 is traversed to indicate thesteps performed during a process may vary from one performance of theprocess to another performance of the process. The flowchart traversalorder may also vary from one process embodiment to another processembodiment. Steps may also be omitted, combined, renamed, regrouped, orotherwise depart from the illustrated flow, provided that the processperformed is operable and conforms to at least one claim.

Examples are provided herein to help illustrate aspects of thetechnology, but the examples given within this document do not describeall possible embodiments. Embodiments are not limited to the specificimplementations, arrangements, displays, features, approaches, orscenarios provided herein. A given embodiment may include additional ordifferent features, mechanisms, and/or data structures, for instance,and may otherwise depart from the examples provided herein.

During a non-blocking request making step 302, an embodiment makes anon-blocking messaging request. Step 302 may be accomplished using callsto an enhanced MPI library, for example.

During an associating step 304, an embodiment associates an asynchronouscallback 224 with a thread context 230 for execution in an enhanced MPIarchitecture, such as by calling an association routine 226, 228. Inparticular, an embodiment may associate 304 a callback routine with arequest 212 as an asynchronous callback to a thread 122.

During a state transitioning step 306, an embodiment transitions athread state 232 between a state that is open to execution to anasynchronous callback and one that is closed to such execution, such asby setting a flag or other indicator 234.

During a callback processing step 308, an embodiment processes acallback 224 by passing control to the callback in an associatedexecution context 230.

During a completion detecting step 310, an embodiment detects that amessaging operation 208 or other messaging request 212 has completed,such as by using familiar MPI completion detection mechanisms.

During a callback queueing step 312, an embodiment queues a callback forexecution 314, such as by placing a pointer or other familiar callbackidentifier in a queue associated with a particular thread 122.

During a non-blocking routine invoking step 316, an embodiment invokes anon-blocking MPI library routine 318. Familiar parameter-passing andcontrol-passing, and other familiar invocation mechanisms can be used,for example.

During a broad association routine invoking step 320, an embodimentinvokes a broad callback association routine 226, which may be one ofthe enhancements to an MPI library 126, for example.

During a narrow association routine invoking step 322, an embodimentinvokes a narrow callback association routine 228, which may be one ofthe enhancements to an MPI library 126, for example.

During a messaging operation performing step 324, an embodiment performsa messaging operation 208, such as a familiar or enhanced operation inan MPI architecture 214.

During a completion-awaiting routine calling step 326, an embodimentcalls (i.e., invokes) a completion-awaiting routine 202, which may beone of the enhancements to an MPI library 126, for example.

During a messaging operation specifying step 328, an embodimentspecifies a messaging operation 208, such as by including a familiar orenhanced operation in a completion condition list 206.

During an engine executing step 330, an MPI progress engine 132executes.

During an interrupting step 332, a thread interrupts execution of aroutine, such as a routine belonging to another thread.

During a control returning step 334, a routine returns control to athread.

During respective steps, an embodiment requests 336 a lock 222, releases340 a lock it has acquired, or otherwise (e.g., gains exclusive accessto a shared resource based on) utilizes 338 a lock 222.

During a complying step 342, an embodiment complies with an MPI standard344, e.g., by invoking a routine 202 which has the same functionsignature 130 as a familiar routine 128 but has a superset of thefamiliar routine's functionality. As used herein, “complies with astandard” and similar phrases mean that a standard's required functionsignatures and minimum required functionality are present; additionalroutines and/or additional functionality in routines that have standardfunction signatures may also be present. For example, one interruptiblewait routine MSMPI_Waitsome_interruptible of an embodiment hereundercomplies with an MPI standard MPI_Waitsome routine in that theinterruptible wait routine behaves like the familiar MPI_Waitsomeroutine if the interruptible wait routine is not interrupted, and alsohas the same function signature as the familiar MPI_Waitsome routine.

During a memory configuring step 346, a memory medium 112 is configuredby an association routine 226, 228, by an enhanced completion-awaitingroutine 202, and/or otherwise in connection with an asynchronouscallback and/or interruptible wait MPI enhancement as discussed herein.

The foregoing steps and their interrelationships are discussed ingreater detail below, in connection with various embodiments.

Some embodiments provide an asynchronous callback enhancement processwhich includes a thread making 302 a non-blocking request to a messagepassing interface library, and associating 304 a callback routine withthe request as an asynchronous callback to the thread.

The thread is a user-mode thread in some embodiments, e.g., in anarchitecture 214 which includes a user-mode MPI library 126. In someembodiments, the thread is not a user-mode thread. For example, kernelmode MPI libraries may be used, and in some environments a distinctionis not made or is not enforced between user mode and kernel mode; thismay occur, e.g., in some embedded systems, or in some systems in whichapplications are fully trusted.

In some embodiments, the process includes the thread transitioning 306state. For example, the thread may transition from a closed state inwhich the thread will not process an asynchronous callback to an openstate in which the thread will process an asynchronous callback, or viceversa. The familiar APC alertable state is an example of an “openstate”, but an embodiment may also use another open state 232.

In some embodiments, the thread (denoted herein as thread X) isaccompanied by another thread, and the process further includes theother thread detecting 310 that the request is complete, and thenqueueing 312 the callback routine for execution by thread X. This may becombined with a state 232 condition, such that a thread other thanthread X causes the callback routine to be queued for execution bythread X when thread X is in the open state.

In some embodiments, making 302 the non-blocking request includesinvoking 316 a message passing interface library non-blocking routine318 which is configured to return a messaging request. The routine 318may be, for example, one of the following: an MPI non-blocking regularsend routine, an MPI non-blocking synchronous send routine, an MPInon-blocking ready send routine, an MPI non-blocking buffered sendroutine, an MPI non-blocking receive routine, an MPI non-blockinggeneralized request routine, an MPI non-blocking I/O request routine.MPI_Issend( ) is an example of an MPI non-blocking synchronous sendroutine, but embodiments are not necessarily limited to a particularversion or implementation of the set of potential MPI libraries 126. Insome embodiments, the non-blocking send routine may be any MPInon-blocking routine that returns an MPI_Request, in that requests 212can be applied to functionality such as connection establishment, bothon the connecting and accepting process.

In some embodiments, the step of associating 304 a callback routine isperformed at least in part by invoking 320 a broad routine 226 which isconfigured to associate callback routines for any of several differentmessage passing interface library routines, each of which is configuredto perform a particular messaging operation. For instance, a routine 226MSMPI_Request_set_apc( ) may be invoked 320 during a process, as opposedto invoking 322 a routine 228 MSMPI_Isend_apc( ).

In some cases, associating 304 a callback routine is performed as aresult of invoking 322 a message passing interface library narrowroutine 228 which is also configured to perform a particular messagingoperation. That is, in some embodiments, the step of associating acallback routine is performed at least in part by invoking a routinewhich is configured both to associate callback routines and also toperform a particular messaging operation. Some embodiments include bothkinds of invocation 320, 322.

In particular, some embodiments include a narrow special APC callbackversion of Isend and Irecv, while others include a broad function bywhich one can set an APC callback for an arbitrary non-blocking request212. With the broad routine, one can initiate other types of sendrequests (Issend, Irsend, etc.), as well as initiate MPI-IO requests,and then associate the APC callback with them. The difference in sourcecode in one particular example would be to execute

MPI_Irecv( ..., &req ) MSMPI_Request_set_apc( &req, cb_fn, cb_stat )instead of MSMPI_Irecv_apc( ..., cb_fn, cb_stat, ..., &req )

In one embodiment, the interface for this broad routine 226 call inMSMPI looks like:

int MPIAPI MSMPI_Request_set_apc( _(——)in MPI_Request request, _(——)inMSMPI_Request_callback* callback_fn, _(——)in MPI_Status* callback_status);

Some embodiments provide a process for asynchronous callback drivenmessaging request completion notification, which includes a user-modethread making 302 a non-blocking request to a message passing interfacelibrary, associating 304 a callback routine with the request as anasynchronous callback to the thread, and queueing 312 the callbackroutine for execution by the thread which made the non-blocking request.

In some embodiments, the process further includes executing 314 thecallback asynchronously in the thread. In some, the queuing 312 stepoccurs in response to another thread detecting 310 that the request iscomplete.

In some embodiments, making 302 the non-blocking request includesinvoking an MPI non-blocking send routine and/or an MPI non-blockingreceive routine. In some, making 302 the non-blocking request includesinvoking an MPI non-blocking I/O request routine and/or an MPInon-blocking generalized request routine.

In some embodiments, the asynchronous callback process includes auser-mode communication thread 216 making 302 a non-blocking request toa message passing interface library, associating 304 a callback routinewith the request as an asynchronous callback to the thread, a user-modeworker thread 218 detecting 310 that the request is complete, andqueueing 312 the callback routine for execution by the communicationthread in response to the detecting step. In some embodiments, theprocess further includes the communication thread transitioning 306 toan open state in which the communication thread is open to processingasynchronous callbacks, and then executing 314 the callbackasynchronously in the communication thread. By way of further example,FIGS. 4 and 5 show compute threads (also called worker threads) makingnon-blocking callback-driven requests, rather than communication threadsdoing so.

Some embodiments provide an interruptible wait process in which acommunication thread 216 makes a wait-some interruptible call to MPI,namely, to an MPI library wait-some routine. Processing occurs in theMPI progress engine 132. At some point, a compute thread 218 interrupts332 the wait-some routine, and the wait-some routine returns 334 to thecommunication thread indicating 210 zero completions.

Some embodiments provide an interruptible wait process enhancement inwhich a thread X calls 326 a completion-awaiting routine of a messagepassing interface library. The called completion-awaiting routine isconfigured (by data and/or instruction structure therein) to returnsuccessfully from a blocked condition after a specified message passingcompletion condition 204 is satisfied. The completion conditionspecifies 328 a list 206 of messaging operation(s) which have not yetbeen completed. For example, thread X could be a communication threadcalling MSMPI_Waitsome_interruptible( ) as in the example of FIG. 7.Continuing the process, a message passing interface library progressengine executes 330 while the completion-awaiting routine is in theblocked condition. A thread Y interrupts 332 the completion-awaitingroutine prior to satisfaction of the completion condition.

In some embodiments, the process further includes the interruptedcompletion-awaiting routine returning 334 control to thread X inconjunction with an indication 210 that none of the listed messagingoperations have completed. In this context, “in conjunction with” meansby use of a parameter, a function result, a global variable, or anotherdata transfer mechanism.

In some embodiments, the completion-awaiting routine 202 is a wait-someroutine, e.g., an enhanced MPI_Waitsome routine of the “wait-family”described above. The wait-some routine has a list 206 of requests, suchas message passing operations, and is configured to return from theblocked condition after a completion of processing occurs on at leastone of the listed requests. Within the list of requests 212, somerequests may be message passing requests (e.g., send/recv), some may beI/O requests (e.g., read/write to file), some may be generalizedrequests, and some may be other requests such as connection relatedrequests (e.g., connect/accept/spawn).

In some embodiments, a thread Y interrupts the completion-awaitingroutine in conjunction with requesting 336 a lock 222 held by the threadX. For instance, a side effect of an MSMPI_Queuelock_acquire routine(discussed herein and illustrated in FIG. 7) is an interruption 332 ofany outstanding MSMPI_Waitsome_interruptible call. The phrase “inconjunction with” here is intended to allow separate procedure calls, aswhen queuelock is a separate routine, and also to allow an integratedapproach, as when sync is not factored out into queuelock but is insteadpart of each MSMPI_wait* function. Some embodiments separate theinterruption functionality from the locking, leaving all synchronizationto the application. This may involve multiple separate locks beingacquired and released in a particular order by the application toprovide FIFO ordering. An alternative approach is use of queuelockfunctionality which encapsulates the FIFO lock management, easing theapplication developer's effort.

More generally, some embodiments utilize 338 a lock 222 for serializingthread calls to the message passing interface library 126. As noted,“lock” is defined broadly herein, with several examples above. Asanother lock example, atomically incrementing an integer can be used asa synchronization mechanism in an implementation of a lock 222.

In some embodiments, an interruptible wait process is performed in anMPI architecture 214 in which thread X is a communication thread 216 andthread Y is one of multiple worker threads 218. In some, the process isperformed in an MPI architecture which complies 342 with at least one ofthe following standards 344: MPI_THREAD_FUNNELED, MPI_THREAD_SERIALIZED.

Some embodiments provide a process for interruptible blocking wait in amessaging architecture. The process includes a thread X calling 326 acompletion-awaiting routine of a message passing interface library 126.The called completion-awaiting routine is configured to returnsuccessfully from a blocked condition after a specified message passingcompletion condition 204 is satisfied. The completion conditionspecifies a list of messaging operation(s) which have not yet beencompleted. A message passing interface library progress engine 132executes 330 while the completion-awaiting routine is in the blockedcondition. At some point, a thread Y interrupts 332 thecompletion-awaiting routine prior to satisfaction of the completioncondition, and the interrupted completion-awaiting routine returns 334control to thread X in conjunction with an indication 210 that thecompletion condition is unsatisfied.

In some embodiments, the completion-awaiting routine completioncondition 204 specifies non-blocking requests 212, and in some itspecifies messaging operations 208. In some embodiments, thecompletion-awaiting routine completion condition 204 specifies that atleast one messaging operation 208 in the list of messaging operation(s)has completed; similarly, it may specify that at least one non-blockingrequest in the list 206 has completed. In some cases, the condition 204specifies that exactly one request/messaging operation in the list hascompleted. In some cases, the condition 204 specifies that multiplerequests/operations in the list have completed, and in some it specifiesthat multiple list members, including every request/messaging operationin the list 206, have completed.

In some embodiments, the indication 210 that the completion condition isunsatisfied specifies how many requests/messaging operation(s) in thelist had completed when thread Y interrupted the completion-awaitingroutine.

In some embodiments, thread Y interrupts 332 the completion-awaitingroutine in conjunction with requesting 336 a lock held by thread X. Insome, the process includes utilizing 338 a FIFO lock for serializingthread calls to the message passing interface library.

In some embodiments, the completion-awaiting routine 202 has the samefunction signature as a standard MPI wait-some routine.

Configured Media

Some embodiments include a configured computer-readable storage medium112. Medium 112 may include disks (magnetic, optical, or otherwise),RAM, EEPROMS or other ROMs, and/or other configurable memory, includingin particular computer-readable media (as opposed to propagated signalmedia). The storage medium which is configured may be in particular aremovable storage medium 114 such as a CD, DVD, or flash memory. Ageneral-purpose memory, which may be removable or not, and may bevolatile or not, can be configured into an embodiment using items suchas enhanced MPI routines 202, callback association routines 226, 228,and code 126, 214 for performing asynchronous callback and/orinterruptible wait processes described herein, in the form of data 118and instructions 116, which are read from a removable medium 114 and/oranother source such as a network connection, to form a configuredmedium. The configured medium 112 is capable of causing a computersystem to perform process steps for transforming data throughasynchronous callback and/or interruptible wait enhancements asdisclosed herein. FIGS. 1 through 3 thus help illustrate configuredstorage media embodiments and process embodiments, as well as system andprocess embodiments. In particular, any of the process steps illustratedin FIG. 3, or otherwise taught herein, may be used to help configure astorage medium to form a configured medium embodiment.

Additional Examples

Additional details and design considerations are provided below. As withthe other examples herein, the features described may be usedindividually and/or in combination, or not at all, in a givenembodiment.

Those of skill will understand that implementation details may pertainto specific code, such as specific APIs and function signatures, andspecific sample programs, and thus need not appear in every embodiment.Those of skill will also understand that program identifiers and someother terminology used in discussing details are implementation-specificand thus need not pertain to every embodiment. Nonetheless, althoughthey are not necessarily required to be present here, these details areprovided because they may help some readers by providing context and/ormay illustrate a few of the many possible implementations of thetechnology discussed herein.

The following discussion and accompanying FIGS. 4-8 are derived in partfrom prototype MSMPI documentation. MSMPI includes MPI library 126 codeimplemented by Microsoft Corporation, and illustrates an MPIarchitecture 214. Aspects of the MSMPI code and/or documentation areconsistent with or otherwise illustrate aspects of the embodimentsdescribed herein. However, it will be understood that MSMPIdocumentation and/or implementation choices do not necessarily constrainthe scope of such embodiments, and likewise that MSMPI and/or itsdocumentation may well contain features that lie outside the scope ofsuch embodiments. It will also be understood that the discussion belowis provided in part as an aid to readers who are not necessarily ofordinary skill in the art, and thus may contain and/or omit detailswhose recitation below is not strictly required to support the presentdisclosure.

Note also that the MSMPI documentation may contain preliminaryinformation or inaccuracies, and may not correctly represent anyassociated Microsoft® product as commercially released. Thedocumentation is provided “as is”, and to the extent permitted by law,Microsoft makes no warranty of any kind, disclaims all express, impliedand statutory warranties, and assumes no liability for any damages ofany type in any connection with the documentation.

Some MSMPI documentation is referred to for convenience under the rubricMPI_THREAD_MOSTLY_SERIALIZED. Some MSMPI functionality may improvescalability and performance of hybrid MPI/multi-thread applications byallowing them to make strong progress without the potential fordeadlock.

In the context of MPI standard concepts MPI_THREAD_FUNNELED,MPI_THREAD_SERIALIZED, and MPI_THREAD_MULTIPLE, noted in a table above,and message flow which spans across nodes as well as threads, alsoabove, the documentation states that interruptible wait functionalityprovides APIs to manage locking semantics between multiple threads, aninterruptible wait function allowing the communication thread to makestrong progress (i.e., spend more time in the MPI progress engine ratherthan polling via MPI_Iprobe or MPI_Test*), and a pair of send andreceive functions that indicate completion via APC callbacks. Theseenhancements can reduce the need for the intra-thread message queues,increase the communication thread's efficiency, and allow computethreads to initiate MPI data transfers directly, thereby yielding betterscalability and better performance than a traditional design.

Some enhancements provide API signatures 130 that allow multiple threadsto serialize access to MPI, allow compute threads to initiate MPImessage transfers directly (bypassing any intra-thread message queueingfor explicit data transfers), and allow compute threads to receivenotifications of message transfer completions directly (bypassing anyintra-thread message queueing for explicit data transfers). As a result,a typical message flow between nodes A, B and threads X (a communicationthread), Y and Z becomes A:Z→B:X→B:Y→A:Z.

In some cases a wait function MSMPI_Waitsome_interruptible is availablewhich allows an application to make strong progress without thepotential for deadlock. To complement this function, and to allow theclient application to interrupt an outstanding call toMSMPI_Waitsome_interruptible, synchronization functions are provided:MSMPI_Queuelock_acquire and MSMPI_Queuelock_release. These allowapplications to easily implement proper serialization to MPI callsbetween threads. A side effect of MSMPI_Queuelock_acquire is that itinterrupts any outstanding MSMPI_Waitsome_interruptible call. Thisdesign allows the messaging thread in the application to make strongprogress, while allowing compute threads to force the messaging threadto break out of the wait in order to initiate data transfers in a timelymanner.

With respect to detailed design in MSMPI, the following is presented.

MSMPI_Lock_queue

The MSMPI_Lock_queue structure is an opaque structure that describes aqueued lock 222. The client application 120 allocates theMSMPI_Lock_queue structure, and passes it to MSMPI_Queuelock_acquire toacquire the queued lock. Those routines initialize the structure torepresent the thread's position in queued lock. The client applicationpasses the same structure to MSMPI_Queuelock_release when releasing thelock, and the structure remains resident in memory as long as the lockis held. Each call to MSMPI_Queuelock_acquire provides a distinctMSMPI_Lock_queue structure as input. The MSMPI_Lock_queue structurecannot be shared between threads.

MSMPI_Queuelock_acquire

This routine acquires the global MSMPI serialization lock. The lockguarantees FIFO ordering for callers, and interrupts any in-progressinterruptible wait calls.

Syntax:

void MPIAPI MSMPI_Queuelock_acquire( _(——)in MSMPI_Lock_queue* queue );

Parameters:

queue

-   -   Pointer to an MSMPI_Lock_queue structure used to represent the        caller in the lock queue. This structure is to remain valid        until a matching call to MSMPI_Queuelock_release, and is not to        be shared between callers.

Return Value:

This function does not return a value.

Remarks:

This function is threadsafe, and can be used to implement properMPI_THREAD_SERIALIZED usage by multiple threads. The lock can onlysafely be acquired recursively in a MPI_THREAD_MULTIPLE environment.

MSMPI_Queuelock_release

Releases the global MSMPI serialization lock 222. The lock guaranteesFIFO ordering for callers.

Syntax:

void MPIAPI MSMPI_Queuelock_acquire( _(——)in MSMPI_Lock_queue* queue );

Parameters:

queue

-   -   Pointer to an MSMPI_Lock_queue structure used to represent the        caller in the lock queue in the matching call to        MSMPI_Queuelock_acquire.

Return Value:

This function does not return a value.

Remarks:

Clients only call this function after a successful call toMSMPI_Queuelock_acquire, and pass the same queue parameter that was usedin the call to MSMPI_Queuelock_acquire.

MSMPI_Request_callback

Defines the function type for the callback invoked via APC when arequest completes.

Syntax:

typedef void (MPIAPI MSMPI_Request_callback)( _(——)in MPI_Status* status);

Parameters:

status [in]

-   -   Pointer to MPI_Status structure specified in the call to        MSMPI_Isend_apc or MSMPI_Irecv_apc, filled in with completion        information for the request.

Return Value:

This function does not return a value.

Remarks:

Clients are to call MPI_Request_free in their callback, or after thecallback is executed, to release the request object. As to obtaining therequest to free, note there is only a single parameter to the callback.If one wants extra context, one can define a structure that contains theMPI_Status as well as any other desired information. One can then useCONTAINING_RECORD to get to this structure and access the other members.

MSMPI_Waitsome_interruptible

Waits for some of the input requests to complete, but can be interruptedby another thread calling MSMPI_Queuelock_acquire.

Syntax:

int MPIAPI MSMPI_Waitsome_interruptible( _(——)in int incount,_(——)inout_ecount(incount) MPI_Request* array_of_requests, _(——)out int*outcount, _(——)out_ecount_part(incount,*outcount) int* array_of_indices,_(——)out_ecount_part(incount,*outcount) MPI_Status* array_of_statuses );

Parameters:

Identical usage as with MPI_Waitsome.

Return Value:

MPI_SUCCESS if the call is successfulOther MPI error value if the call fails.

Remarks:

This function behaves in some regards identically to MPI_Waitsome, withthe additional behavior that the function can return MPI_SUCCESS withoutcount set to 0 if the call was interrupted by another thread calling

MSMPI_Queuelock_acquire.

Callers are expected to call MSMPI_Queuelock_acquire before calling thisfunction, and must call MSMPI_Queuelock_release when this functionreturnsoutcount==0 in order to allow other threads to run.

MSMPI_Isend_apc

Starts a non-blocking send that will complete via an APC callback to therequesting thread.

Syntax:

int MPIAPI MSMPI_Isend_apc( _(——)in void* buf, _(——)in int count,_(——)in MPI_Datatype datatype, _(——)in int dest, _(——)in int tag,_(——)in MSMPI_Request_callback* callback_fn, _(——)in MPI_Status*callback_status, _(——)in MPI_Comm comm, _(——)out MPI_Request* request );

Parameters:

buf, count, datatype, dest, tag, comm, request

Identical usage as with MPI_Isend.

callback_fn [in]

The callback function to invoke when the request completes.

callback_status [in]

-   -   The status to pass to the callback when the request completes,        indicating completion information.

Return Value:

MPI_SUCCESS if the call is successfulOther MPI error value if the call fails.

Remarks:

Applications are not to exit threads that have pending MSMPI_Isend_apcrequests outstanding until the request completes and the APC callback isinvoked. Doing so leaks a reference to the requesting thread until theapplication terminates, and will cause an error to be reported whenMSMPI fails to queue the APC callback. Applications can canceloutstanding requests using MPI_Request_cancel as with any othernon-blocking request. Applications that make blocking MPI calls from athread with APC requests outstanding may execute the APC callback whilein the context of the blocking MPI call. MSMPI will support applicationsmaking MPI calls from the APC callback. However, applications are to becareful about nesting APC callbacks too deeply and running out of threadstack space.

MSMPI_Irecv_apc

Starts a non-blocking receive that will complete via an APC callback tothe requesting thread.

Syntax:

int MPIAPI MSMPI_Irecv_apc( _(——)out void* buf, _(——)in int count,_(——)in MPI_Datatype datatype, _(——)in int source, _(——)in int tag,_(——)in MSMPI_Request_callback callback_fn, _(——)in MPI_Status*callback_status, _(——)in MPI_Comm comm, _(——)out MPI_Request* request );

Parameters:

buf, count, datatype, source, tag, comm, request

Identical usage as with MPI_Irecv.

callback_fn [in]

The callback function to invoke when the request completes.

callback_status [in]

-   -   The status to pass to the callback when the request completes.        The status is updated with completion information before the        callback is invoked. Memory for this structure is to remain        valid until the request completes.

Return Value:

MPI_SUCCESS if the call is successfulOther MPI error value if the call fails.

Remarks:

Applications are not to exit threads that have pending MSMPI_Irecv_apcrequests outstanding until the request completes and the APC callback isinvoked. Applications can cancel outstanding requests usingMPI_Request_cancel as with any other non-blocking request. Applicationsthat make blocking MPI calls from a thread with APC requests outstandingmay execute the APC callback while in the context of the blocking MPIcall. MSMPI will support applications making MPI calls from the APCcallback. However, applications are to be careful about nesting APCcallbacks too deeply and running out of thread stack space.

As to logging, the foregoing MSMPI APIs will follow the same tracinglogic as existing MPI APIs.

With further attention to asynchronous callback enhancements, some MSMPIdocumentation notes in the context recited above regarding MPI standardsupport for multiple levels of threading in MPI implementations(MPI_THREAD_FUNNELED etc.) that one threading pattern implemented byseveral HPC applications has a single messaging thread in charge ofmaking MPI calls (effectively MPI_THREAD_FUNNELED.) The application thenimplements message queues between compute threads and the messagingthread to allow incoming messages to be dispatched to compute threads aswell as to allow compute threads to queue requests to be sent andreceived via MPI by the messaging thread.

To effectively overlap communication with computation, some applicationsuse non-blocking MPI requests, generally initiated via MPI_Isend andMPI_Irecv. The application then explicitly waits for completion of therequests via a call to MPI_Wait, MPI_Test, or one of their variants.

Some embodiments introduce non-blocking send and receive functions 228,MSMPI_Isend_apc and MSMPI_Irecv_apc, that avoid waiting for completionexplicitly. Instead, any thread making progress that completes anoutstanding APC request causes an APC to be queued 312 to the initiatingthread. The APC executes outside of the context of any MPI call wheneverthe requesting thread enters an alertable state 232.

As further examples, one has the following normal use case and anerroneous use case:

Normal Use Case:

Thread 1 Thread 2 MSMPI_Isend_apc( req1 ) . . . SleepEx( INFINITE,alertable = MPI_Iprobe TRUE ) . . . [MPI_Iprobe makes progress on req1,it completes, and an APC is queued. APC runs, then SleepEx returns . . .MPI_Request_free( req 1 )

Erroneous Use Case:

Thread 1 Thread 2 MSMPI_Isend_apc( req1 ) . . . ExitThread MPI_Iprobe[MPI_Iprobe makes progress on req1, it completes, and an APC fails to bequeued because thread exited. MPI_Iprobe fails . . .

As to error handling, standard MPI error handling can be applied tothese MSMPI functions. An error queuing the APC would result in an errorbeing reported by the function that tried to queue the APC (the functionthat detected the request completion, generally MPI_Wait or one of itsderivatives.)

As to logging, the enhanced APIs would be integrated into the usualMSMPI tracing. As to setup and upgrade, existing MPI interfaces remainunchanged in behavior. No backward compatibility issues arise, as noexisting APIs are modified in this particular implementation.

As to performance and scale, because the requesting thread is notifiedimplicitly by MSMPI queuing an APC, the messaging thread avoids explicitwait for such requests. The messaging thread is also relieved of theduty of dispatching request completion notifications to the appropriatethread, reducing code complexity, and allowing it to spend more of itsCPU cycles making progress on other MPI requests. Overall message ratesfor a multi-threaded app that uses the new functions may improve. Someadditional logic is present when requests are creating (zero-ing anadditional member), and an extra conditional branch in the requestcompletion path to queue the APC. Both of these would impact non-APCrequests. Request creation and completion code paths are modified, sotesting (e.g., pingpong and message rate tests) could be done to seewhether applications using the standard MPI routines regress.

FIGS. 4 through 8 illustrate usage of MSMPI routines. As an additionalexample, the following usage model is provided:

Communication Thread Compute Thread MSMPI_Queuelock_acquire . . .(compute) MPI_Irecv( req1 ) . . . (compute)MSMPI_Waitsome_interruptible( . . . (compute) req1 ) . . . (in progressengine) MSMPI_Queuelock_acquire [wait is interrupted] . . . (spinning)MSMPI_Queuelock_release . . . (spinning) MSMPI_Queuelock_acquireMPI_Send . . . (spinning) MSMPI_Irecv_apc( req2 ) . . . (spinning)MSMPI_Queuelock_release MSMPI_Waitsome_interruptible( SleepEx( INFINITE,alretable = req1 ) TRUE ) [req1 completes] . . . (sleeping)MSMPI_Queuelock_release . . . [dispatch req1] . . .MSMPI_Queuelock_acquire . . . MPI_Irecv( req1 ) . . .MSMPI_Waitsome_interruptible( . . . req1 ) [req2 completes, queues APCto thread . . . 2] . . . (in progress engine) APC runs . . . (inprogress engine) MSMPI_Queuelock_acquire [wait is interrupted] . . .(spinning) MSMPI_Queuelock_release . . . (spinning)MSMPI_Queuelock_acquire MPI_Request_free( req2 ) . . . (spinning)MSMPI_Queuelock_release MSMPI_Waitsome_interruptible( . . . (processreceived data) req1 )

In this example, reg1 serves to handle incoming requests from otherprocesses. This would have a different tag from req2, which serves toreceive the response to the request sent by the MPI_Send call. Thecommunication thread does not use the req2 handle, so one avoids theintra-thread communication. Note that the [dispatch req1] step could bedone by queueing one's own APC to that thread if the target is a computethread that might be sleeping. Note also that the queue parameter toMSMPI_Lockqueue_acquire is to be passed to the correspondingMSMPI_Lockqueue_release call.

From an architectural perspective, one queue lock implementation willparallel the kernel's in-stack queued spinlock functionality.Specifically, there will be a global ‘tail’ pointer for the queue, setto NULL if the queue is empty, or to the address of the MSMPI_Lock_queueentry currently at the tail. This allows usingInterlockedExchangePointer to test for NULL as well as queueing entries.When a thread calls MSMPI_Waitsome_interruptible, a global flag is setindicating that a thread is in that function. This allows a future callto MSMPI_Queuelock_acquire to signal the thread by queueing a completionto the progress engine IOCP, interrupting the wait call. APCs are notused for waking up the thread from the completion port because one mightbe alerted due to an APC request completing, and then execute the APCcallback while still in the context of an MPI call. Care will be takento delay executing user APC requests if one ever moves to using APCsinternally until unwinding.

A challenging code flow is the following:

MSMPI_Queuelock_acquire MSMPI_Isend_apc( req1 ) MPI_Irecv( req2 )MSMPI_Waitsome_interruptible( req2 ) [req1 completes, queue APC] [APCexecutes, causes wait to be aborted] MSMPI_Queuelock_release SleepEx(INFINITE ) [waiting for req1 APC] ← DEADLOCKAlternatively, if the user's APC attempts to perform MPI operations, andcalls MSMPI_Queuelock_acquire, it will deadlock.

In one example, MSMPI_Queuelock_acquire implements the followingpseudocode:

MSMPI_Queuelock_acquire( queue ) { queue−>next = NULL oldTail =InterlockedExchangePointer( &g_pLockTail, queue ) if( oldTail == NULL )return MPI_SUCCESS; queue−>waiting = TRUE; oldTail−>next = &queue if(InterlockedBitTestAndSet( g_interruptWaitFlag ) && g_fWaiter == TRUE ) {ExPostCompletion( EX_WAIT_INTERRUPT ) } loop while( queue−>waiting )return MPI_SUCCESS }

In this example, MSMPI_Queuelock_release implements the followingpseudocode:

MSMPI_Queuelock_release( queue ) { if( queue−>next == NULL ) { oldTail =InterlockedCompareExchangePointer( &g_pLockTail, NULL, queue ) if(oldTail == queue ) return loop while( queue−>next == NULL ) }queue−>next−>waiting = FALSE; }

In this example, the interruptible progress loop implements thefollowing pseudocode:

do { while( spin limit not reached ) { if SHM request completion, returnrequest status if ND request completion, return request status ifslow_tick or interrupted, check IOCP for request completion, and iffound return status if interrupted, return MPI_ERR_PENDING } if(blocking or interruptible ) { enable SHM completion notification enableND completion notification GetQueuedCompletionStatus( INFINITE ) if(interruptible && key == EX_WAIT_INTERRUPT ) return MPI_ERR_PENDING } }while( no completion && (blocking || interruptible) );

In this example, MSMPI_Waitsome_interruptible implements the followingpseudocode:

poke progress engine to attempt to make minimal progress g_fWaiter =TRUE if( g_interruptWaitFlag is set ) { clear g_interruptWaitFlag clearg_fWaiter outcount = 0 return MPI_SUCCESS; }

-   normal waitsome logic, except blocking progress call is replaced    with interruptible progress loop:

if( interruptible progress loop == MPI_ERR_PENDING ) { clearg_interruptWaitFlag clear g_fWaiter outcount = 0 return MPI_SUCCESS; }

CONCLUSION

Although particular embodiments are expressly illustrated and describedherein as processes, as configured media, or as systems, it will beappreciated that discussion of one type of embodiment also generallyextends to other embodiment types. For instance, the descriptions ofprocesses in connection with FIGS. 3 through 8 also help describeconfigured media, and help describe the operation of systems andmanufactures like those discussed in connection with other Figures. Itdoes not follow that limitations from one embodiment are necessarilyread into another. In particular, processes are not necessarily limitedto the data structures and arrangements presented while discussingsystems or manufactures such as configured memories.

Not every item shown in the Figures need be present in every embodiment.Conversely, an embodiment may contain item(s) not shown expressly in theFigures. Although some possibilities are illustrated here in text anddrawings by specific examples, embodiments may depart from theseexamples. For instance, specific features of an example may be omitted,renamed, grouped differently, repeated, instantiated in hardware and/orsoftware differently, or be a mix of features appearing in two or moreof the examples. Functionality shown at one location may also beprovided at a different location in some embodiments.

Reference has been made to the figures throughout by reference numerals.Any apparent inconsistencies in the phrasing associated with a givenreference numeral, in the figures or in the text, should be understoodas simply broadening the scope of what is referenced by that numeral.

As used herein, terms such as “a” and “the” are inclusive of one or moreof the indicated item or step. In particular, in the claims a referenceto an item generally means at least one such item is present and areference to a step means at least one instance of the step isperformed.

Headings are for convenience only; information on a given topic may befound outside the section whose heading indicates that topic.

All claims and the abstract, as filed, are part of the specification.

While exemplary embodiments have been shown in the drawings anddescribed above, it will be apparent to those of ordinary skill in theart that numerous modifications can be made without departing from theprinciples and concepts set forth in the claims, and that suchmodifications need not encompass an entire abstract concept. Althoughthe subject matter is described in language specific to structuralfeatures and/or procedural acts, it is to be understood that the subjectmatter defined in the appended claims is not necessarily limited to thespecific features or acts described above the claims. It is notnecessary for every means or aspect identified in a given definition orexample to be present or to be utilized in every embodiment. Rather, thespecific features and acts described are disclosed as examples forconsideration when implementing the claims.

All changes which fall short of enveloping an entire abstract idea butcome within the meaning and range of equivalency of the claims are to beembraced within their scope to the full extent permitted by law.

What is claimed is:
 1. A process comprising: receiving from a requestingprogram a first call which is made to a messaging routine, the firstcall providing at least one request for at least one requested messagingoperation to be performed by the messaging routine; permitting therequesting program to continue execution prior to completion of the atleast one requested messaging operation rather than blocking executionof the requesting program in response to the first call; interruptingthe messaging routine prior to completion of the at least one requestedmessaging operation in response to a second call which is made from therequesting program to a synchronization routine; and wherein calling thesynchronization routine has an effect of interrupting any outstandingcall to the messaging routine.
 2. The process of claim 1, wherein therequesting program is an application program.
 3. The process of claim 1,wherein the requesting program is a kernel program.
 4. The process ofclaim 1, wherein the requesting program has at least a first thread anda second thread, the first thread makes the first call, and the secondthread makes the second call.
 5. The process of claim 1, wherein thefirst call provides a plurality of requests for a plurality of requestedmessaging operations to be performed by the messaging routine.
 6. Theprocess of claim 1, wherein the messaging routine has a functionsignature which complies with a Message Passing Interface Forumspecification.
 7. The process of claim 1, wherein the messaging routineis a completion-awaiting routine of a library.
 8. A computer-readablestorage medium configured with data and with instructions that whenexecuted cause one or more processors to perform a process, the processcomprising the steps of: receiving from a requesting program a firstcall which is made to a messaging routine, the first call providing atleast one request for at least one requested messaging operation to beperformed by the messaging routine; permitting the requesting program tocontinue execution prior to completion of the at least one requestedmessaging operation rather than blocking execution of the requestingprogram in response to the first call; interrupting the messagingroutine prior to completion of the at least one requested messagingoperation in response to a second call which is made from the requestingprogram to a synchronization routine; wherein an application programinterface containing the synchronization routine has functionalitysupporting serialization of calls to the messaging routine.
 9. Thestorage medium of claim 8, wherein the requesting program includes atleast one of the following: an application program, a kernel program, amultithreaded program.
 10. The storage medium of claim 8, wherein thefirst call provides a plurality of requests for a plurality of requestedmessaging operations to be performed by the messaging routine, and theprocess further comprises the messaging routine indicating an extent towhich the requested messaging operations had completed when themessaging routine was interrupted.
 11. The storage medium of claim 8,wherein the first call is made by a thread, and the process furthercomprises associating a callback routine with the at least one requestas an asynchronous callback to the thread.
 12. The storage medium ofclaim 8, wherein the application program interface containing thesynchronization routine is part of a message passing interface library.13. The storage medium of claim 8, wherein the application programinterface containing the synchronization routine includes at least oneof the following: a non-blocking send routine, a non-blocking receiveroutine, a non-blocking I/O request routine.
 14. A computer systemcomprising: at least one logical processor, each logical processor inoperable communication with a memory, the memory comprising at least onephysical storage device; at least one program which resides in thememory; a messaging routine which resides in the memory; asynchronization routine which resides in the memory; an applicationprogram interface which contains the synchronization routine; andwherein code executes in the system in which the program makes a call tothe messaging routine providing at least one request for at least onerequested messaging operation to be performed by the messaging routine,the program continues execution prior to completion of the at least onerequested messaging operation rather than being blocked by the call tothe messaging routine, the program subsequently makes a call to thesynchronization routine, which results in an interruption of themessaging routine prior to completion of the at least one requestedmessaging operation, the call to the messaging routine is serialized,and execution of the program continues.
 15. The system of claim 14,further comprising a display controlled at least in part by the program,and wherein the system comprises at least one of the following: a cellphone, a mobile phone, a mobile device.
 16. The system of claim 14,wherein the system comprises a tablet.
 17. The system of claim 14,wherein the program comprises kernel software.
 18. The system of claim14, wherein the messaging routine and the synchronization routine arepart of a message passing interface library in the system.
 19. Thesystem of claim 14, wherein the messaging routine is a first messagingroutine, and the synchronization routine utilizes a lock to serializeaccess to the first messaging routine and at least one other messagingroutine in the system.
 20. The system of claim 14, wherein the system isa multiprocessor system which includes a general purpose processor andalso includes at least one of the following: a graphics processor, asignal processor.